Skip to content

BDMS 159: well transfers updates, water level transfer updates, & other revisions#180

Merged
jirhiker merged 45 commits into
stagingfrom
jab-waterlevels-thing-transfer-updates
Oct 10, 2025
Merged

BDMS 159: well transfers updates, water level transfer updates, & other revisions#180
jirhiker merged 45 commits into
stagingfrom
jab-waterlevels-thing-transfer-updates

Conversation

@jacob-a-brown

@jacob-a-brown jacob-a-brown commented Oct 9, 2025

Copy link
Copy Markdown
Contributor

Why

This PR addresses the following problem / context:

  • well and water level transfer scripts need to be updated for the new and revised models
  • well depth, hole depth, and well casing depth need to be validated against each other if information is present
  • MeasuredBy from the WaterLevels table needs to be mapped to contacts in Ocotillo

How

Implementation summary - the following was changed / added / removed:

  • update well transfer script to incorporate new and revised fields.
    • to enable effective_start to be added to the location association model I needed to add the well manually and not through add_thing
  • updated MeasuredBy mappings with Ethan's and Stacy's input. If a record cannot be mapped the new name is what is from NM_Aquifer and both organization and role are set to Unknown. We'll know if that record comes from NM_Aquifer since the associated nma_pk_waterlevels field will be populated.

Notes

Any special considerations, workarounds, or follow-up work to note?

  • this incorporates the changes made in PR NO TICKET: thing update and fix #179 (eager loading). because of eager loading .unique() needed to be added to select statements in the transfer script
  • this incorporates changes that have been made to the feature branch and are not yet present in staging
  • because the same session is used throughout the transfer process, when data is added to the DB sometimes SQLAlchemy stores that data in memory (cached), so when I retrieve a record it doesn't always reflect the data types in the database. This is why I have to expunge and refresh locations from the session. Otherwise the point field is sometimes a string and the latlon property fails (it's not retrieved as a WKBElement and therefore has no x or y attributes)

WORK TO BE DONE

This PR is to catch the transfer scripts up to the current state of the models (for the most part). There are additions and amendments we want to make, such as well metadata fields. To keep this PR focused and to preclude complications that will arise from model changes, revisions will occur in future PRs. As of now these are (as I recall, there is likely more we want to do):

  • updating the StatusHistory table in the well transfer script if a well is marked as destroyed
  • adding nma_phone to the Phone table for access to invalid legacy data (mostly phone numbers that are 7 and not 10 digits)
  • adding organizations from OwnersData to enable owners' organization field to be valid when transferring data from OwnersData (NM_Aquifer) to Contact (Ocotillo)
  • adding well metadata fields when warranted and after careful consideration (e.g. sample_ok, formation, open_well_logger_ok)
  • adding tables for WellPurpose and WellCasing since a well may serve multiple purposes and well casing may be made of many materials
  • adding the polymorphic notes table

jirhiker and others added 30 commits October 1, 2025 11:38
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.44444% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
services/util.py 37.50% 5 Missing ⚠️
schemas/location.py 87.50% 1 Missing ⚠️
services/thing_helper.py 85.71% 1 Missing ⚠️
Files with missing lines Coverage Δ
api/search.py 97.77% <100.00%> (ø)
api/thing.py 98.46% <ø> (ø)
db/location.py 95.00% <100.00%> (ø)
db/thing.py 100.00% <100.00%> (ø)
schemas/contact.py 100.00% <100.00%> (ø)
schemas/thing.py 99.15% <100.00%> (ø)
services/geospatial_helper.py 74.46% <100.00%> (ø)
tests/conftest.py 98.59% <100.00%> (ø)
tests/test_location.py 100.00% <ø> (ø)
tests/test_observation.py 95.65% <ø> (ø)
... and 5 more

@jirhiker jirhiker merged commit 75a2193 into staging Oct 10, 2025
3 checks passed
@TylerAdamMartinez TylerAdamMartinez deleted the jab-waterlevels-thing-transfer-updates branch February 5, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants